Application of Kernel Trick to Fuzzy c-Means with Regularization by K-L Information

Support vector machines (SVM), kernel principal component analysis (KPCA), and kernel Fisher discriminant analysis (KFD), are examples of successful kernel-based learning methods. By the addition of a regularizer and the kernel trick to a fuzzy counterpart of Gaussian mixture density models (GMM), this paper proposes a clustering algorithm in an extended high dimensional feature space. Unlike the global nonlinear approaches, GMM or its fuzzy counterpart is to model nonlinear structure with a collection, or mixture, of local linear sub-models of PCA. When the number of fearture vectors and clusters are n and C respectively, this kernel approach can find up to C × n nonzero eigenvalues. A way to control the number of parameters in the mixture of probabilistic principal component analysis (PPCA) is adopted to reduce the number of parameters. We apply the kernel trick to the clustering method in a high dimensional feature space. The algorithm provides a partitioning with flexible shape of clusters in the original input data space.